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Abstract — A robust (deterministic) filtering approach 
to the problem of optimal sensor selection is considered 
herein. For a given system with several sensors, at each 
time step the output of one of the sensors must be chosen 
in order to obtain the best state estimate. We reformulate 
this problem in an optimal control framework which can 
then be solved using dynamic programming. In order to 
tackle the numerical computation of the solution in an 
efficient manner, we exploit the preservation of the min- 
plus structure of the optimal cost function when acted upon 
by the dynamic programming operator. This technique 
yields a grid free numerical approach to the problem. 
Simulations on an example problem serve to highlight the 
efficacy of this generalizable approach to robust multi- 
sensor state estimation. 

I. Introduction 

Various problems of practical interest involve the task 
of estimating the state of the system of interest under un- 
der the presence of a number of sensors. For example, a 
'sensor fusion' problem involves determining an optimal 
way to combine the measurements from multiple sensors 
each of which may measure a subset of the states of the 
system. This is, in essence, a state estimation problem. 

A more involved problem arises when the state es- 
timation strategy is given the freedom to dynamically 
select the set of sensors used. Such a 'sensor scheduling' 
problem can be used to model scenarios encompassing 
robust estimation under potential sensor faults; in addi- 
tion this framework may be used to preselect a set of 
sensors to be used, based on constraints on the avail- 
able communication bandwidth or energy limitations on 
transmitting sensed information. 

In the domain of optimal state estimation, there 
are two distinct approaches to estimation/filter design 
which have been applied in the design of estimators for 
systems subjected to process and measurement noise. 
The most well known approach to filter design is 
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the Kalman filter developed for linear systems in the 
1960's [13]. The Kalman filter uses the statistics of the 
noise/measurement processes in order to compute the 
weightings on the measurement error, used to update 
the state estimate, as new measurements are observed. 
An alternative approach is the minimum energy filtering 
technique developed by Mortensen [14]. This interpre- 
tation views the optimal state estimation problem as 
an optimal control problem wherein the objective is to 
minimize the energy of the noise process required to 
explain the observations. The resulting filter obtained 
via both these approaches for the linear system case with 
white noise processes (and a quadratic energy function 
with L 2 noise processes) turns out to be the Kalman 
filter [6], [15], [16]. 

The articles [2], [3] address the sensor scheduling 
problem in the case of linear systems with Gaussian 
input noise. An alternative approach for optimal sensor 
scheduling in nonlinear systems, formulated via impos- 
ing a cost on the switching of sensor choice and the error 
in estimation due to this choice, was undertaken in [1]. 
The solution for this case depends on the solution of a 
Riccati differential equation arising from the choice of 
sensors, and can be solved offline. Generalized versions 
of this problem for the case of uncertain systems with 
linear (albeit time varying) dynamics using integral 
quadratic constraints to model the uncertainty of the sys- 
tem were studied in [4]. The latter uses a deterministic 
filtering interpretation of the estimator (c.f. [5], [6]). 

In this article we study an approach to solving prob- 
lems of sensor scheduling for systems with linear dy- 
namics which involves solving a modified version of the 
deterministic filtering problem. The organization of the 
article is as follows. In Sec. [H] we formulate the estima- 
tion problem as an associated optimal control problem 
which may be solved via dynamic programming. Hence, 
the methods employed herein generalize deterministic 
filtering to deal with sensor selection strategies. In 



Sec. [In] we describe how the Hamilton-Jacobi-Bellman 
(HJB) equation may be solved using a new class of com- 
putational approaches to solving such partial differential 
equations. The underlying idea involves modeling the 
value function as an element of a semi-convex function 
space whose structure is preserved under the dynamic 
programming operator. Such a technique belongs to a 
class of recently developed methods termed max-plus 
methods which have led to intriguing advancements in 
computational tractability for optimal control problems 
in several domains [7]— [ 11]. The techniques introduced 
herein can be extended to certain classes of nonlinear 
dynamics and nonlinear output functions as well [12]. 



In Sec. IV we demonstrate the application of the theory 
developed, using an example system with five sensors. 
We then conclude with a description of problems of 
interest and avenues for further exploration in Sec. [V] 

II. System description 

In this article we consider the class of systems with 
discrete time dynamics of the form 



x k+1 = Ax k + Bu k + w k , 



x k ,w k € 



(1) 



where x k denotes the system state at a time instant k 
and w k is a disturbance signal. The output equation for 
the system with M sensors is 



V{ = C 3 x k +r] J k , 



i6{l,2,...M}. 



where ry^, is measurement noise in the j th sensor at time 
step k. We assume that the M sensors above are such 
that the pairs (A, C J ) are observable for each j. 

In the results to be presented in this article, we make 
use of a time reversed representation of the dynamics 
rather than the time incremental version in Q. This 
time-inverted system has the model 



x k+1 = Ax k + Bu k + w k , 



x k ,w k e 



(2) 



and we denote the solution to this dynamics at any time 
step k given the state at a different time instant j, a 
control signal u and a disturbance w as 

X(j\xk+i = x,u(-),w). 

For this M sensor system we pose the dynamic sensor 
selection problem as a modification of the deterministic 
filtering problem as follows. We commence by modeling 
the choice of sensors at each time step as an element A 
of the unit ball S M . Specifically, if only one sensor say 
the q th sensor) is chosen, then the vector A is an M 
dimensional vector with 1 at the q-th element and at 
the others. Hence A has a probabilistic interpretation in 
terms of the relative belief in each sensor's accuracy. 



For any particular disturbance signal realization and 
the control signal A(-) for the choice of sensor, we 
consider a cost function of the form 



J(A, w,x) = | £ ( w k) ( w k) + P(0) - ioll 

K M 



+ 



(3) 

k=0 j=0 

with weighting matrices H and L on the measurement 
error and state estimation error respectively. Here Xq 
is the assumed value of the initial state of the system 
and x(k) is the estimated state of the system at time 
k under the dynamics |2]). Hence, the cost function 
penalizes the disturbance, deviations from the system 
dynamics (in terms of the deviation in the initial states 
and the measurement error). This form of the cost 
function differs from the standard form of the robust 
(deterministic) filter [5] in the following manner. Here 
the choice of the sensors is incorporated into the cost of 
using the final term in ([3]). Note that the choice Xj of 
sensor beliefs is allowed to vary at each time step. We 
may also penalize changes in the choice of sensors as 
was done in [1] or study a one time step finite horizon 
control problem as in [4]. 

The optimal cost function for this problem is 

V k (x) := inf J k (x, X,w), 
Aes M (-) 

where S AI (-) and W are the piecewise continuous 
signals taking up values in S M and K respectively. The 
value function at any time k leads directly to the optimal 
state estimate via the relation 

x* k := arg inf V k {x). 

Now, we apply the dynamic programming principle 
from optimal control theory to obtain 



V k+1 (x) = inf I V k (X(k\x k+1 = x, u, w) 

M 

E \\vi + i -c j xf H xi\ 



\es R 



3=0 



This equation, can be cast in the form of an operator S 
acting on the value function as follows: 



V k+1 (x) 



V k 



(x). 



(4) 



Standard approaches to solve for the optimal cost 
function (value function) involve the solution of the 
associated Hamilton-Jacobi-Bellman partial differential 
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equation which is the infinitesimal limit of the above 
equation. However, this leads to issues of large compu- 
tational complexity, especially in cases with nonlinear 
dynamics and a higher number of dimensions of the 
state space. In this work, we apply a technique which 
has shown much promise in enabling the solution to 
certain classes of optimal control/ optimal filtering prob- 
lems [9], [19] - for both linear and nonlinear systems. 
This approach makes use of the fact that the dynamic 
programming operator, Q above, is linear on the space 
of semi-convex functions. Thus if we prove that the 
structure of the value function is preserved under the 
dynamic programming operator and the parameterization 
can be done independent of the state space (i.e. the 
parameters of the value function do not depend on the 
point in space at which the value function is evaluated), 
then we can obtain a numerically efficient technique 
to solve optimal control problems which can be cast 
into this form by repeated applications of the dynamic 
programming operator (while storing only the parameter 
values). Further background details and specific applica- 
tions of this concept to various problems can be found 
in [7], [11], [17]-[21]. 

III. The min-plus expansion and the 

PROPAGATION OF THE COST FUNCTION 

In this section we demonstrate that under one time 
step of propagation by the dynamic programming op- 
erator (as in Q) a specific form of the value function 
is preserved. As indicated previously, this feature leads 
to an efficient approach to solve for the value function 
(and hence, to the state estimate). The value function 
thus determined may be used to obtain the optimal state 
estimate. We start with a description of the solution 
technique for the cost function 

A. Min-plus approach to solving the optimal control 
problem 

Motivated by the terminal form of the value function 



V (x) 



\x - X \\l , 



we assume that the value function at any time step k 
can be written as follows 

V k (x) = [x T Q a a x + 2Qtx + Q«}. (5) 
aer k 

Here T k is the index set at the time instant k and the 
notation (J) denotes the minimization operation over the 
index set. We assume that the quadratics associated with 
the index set are convex, leading to a well defined state 
estimate. 



Theorem 3.1: Given the form |5]l of the value func- 
tion at time k, the propagation under the dynamic 
programming operator Q leads to a preservation in 
structure of the value function viz. there exists a set 
Tfc+i and spatially invariant parameters R^, i?f , 
for /3 € T k+ i such that the value function at time k + 1 
is 

V k+ i(x)= [x T Qf ) a x + 2Qlx + Q^ c ). 

Proof: Assuming the form of the value function in 
(5 i, we note that the action of the propagation equation 
(4 1 leads to following recursion 

V k +i{x\u) = 

inf <^ V k (X(k\x k+1 = x, u k+1 = u, w k+1 = w)) . . . 
\es M 



inf 



{v k {Ax + Bu + w) + . . 



w 



M 



3=0 



(6) 

Using the form of the value function <j3j in |6]) we have 

V k+ i{x\u) = 

inf | \{Ax + Bu + w) T QZ{Ax + Bu + w)- 
2Q%(Ax + Bu + w) + Q"} + \\w\\ 2 + ... 

M 

X Hi.: (■'■>■ 7/ A'} 



(7) 



3=0 



We note that for each j, since w and \\y^ — C^x\\ are 
independent, therefore the optimal value of A can be 
determined from the last term in |7]). Specifically the 
optimal value of a linear function of A on a compact set 
is given by a point on the boundary of the compact set. 
This leads to one of the sensors being used in one time 
step with the others being turned off (in the case where 
the sensor dynamics can be altered at each time step). 
The optimal argument A* has one element Xj equal to 
1 with all other elements of A set to 0, where 



j := argmin\\y{ +1 - C J x\\\ 

3 

For simplicity of notation we denote 

z := Ax + Bu. 



(8) 



(9) 
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Note that for ever value of the state x, the value of 
w which minimizes the quadratic portion of the cost 
function dependent on w can be obtained analytically 
as follows. For a triple (Q%, Q b , Q")> the corresponding 
w* which minimizes the term 



The optimal state estimate at time step N is defined as 



(z + w) T Q%(z + w) + 2Q«(z + w)+ 

Qc+\H\ 2 



(10) 



in the value function, can be shown to have the analytic 
form 



w 



(I+Q a aY 



(ID 



Hence, for every a e there exists a disturbance 
value w* which can be optimally chosen to minimize the 
quadratic form ( in w) in the cost function. Substituting 
this value of w* in ( fTO) and using |9]), we obtain a 
quadratic form in x which can be written as 



x T P:x + 2P b a x + P?, 



(12) 



where from |9]), (JTOf and ( fTT| it can be seen that P", 
P b , P" are obtained from the coefficients of quadratic 
terms in x in ( fTO) and using the term Q", Q b , Q" 
respectively. Thus using <|8j and ([12} the value iteration 
in (|7]i simplifies to 

V k+1 (x\u) = x T P°x + 2P?x + P? + 



aer k 



®\\vL+i-c 3 *\\% (13) 



Creating a new index set Tk+i =FkxA4 and combining 
coefficients of equal powers of x, we can rewrite ( p"3j ) 

as 

V k+1 (x\u)= x T R a x + 2R^x + R^ c , 
/3er fc+1 

where it can be seen that R@ is obtained from P" and 
terms from a subset of the C J . This completes the proof 
of the desired result. ■ 

B. Generating the state estimate 

We now indicate how the value function obtained 
above is used to compute the optimal state estimate. 
Starting with the terminal cost, we apply the propagation 
procedure for N time steps (i.e., for the entire duration 
of the estimation process) to obtain a value function of 
the form 

V N {x\u{-)) = x T R2x + 2R2x + R2, (14) 
7er„ 



x* N = argmin V N (x\u(-)). 



(15) 



Now given ( |14) , and an unconstrained state space, we 
can solve for the state estimate ( fT5] > by determining the 
minimum for each quadratic in the basis expansion ( [14"} . 
Thus 



n* N = axgmmV N (x\u(-)), 



where 



X, 



N 



v 7 g rv}. 



The set Ajv can be seen to be the set of the minima of 
the set of convex quadratic functions in the expansion 

<£!■ 

IV. Example 

In this section we validate the theory developed thus 
far in this article, using an example problem as follows. 
We consider a first order linear system with forward time 
dynamics 



x k +i = AfX k + BfU k + B w ftv k , 
A f = 0.7, B f = 0.4, B w f = 



-0.7. 



Without loss of generality, the value of B w f has been 
chosen in order to ensure the form of the backward 
dynamics in Q i.e., the coefficient of the disturbance 
term becomes 1. We assume the presence of five sensors 
with linear output maps 



J 

Vk 



C k Xk, 



where the C j belong to the set {1.5,-2,1.7,3.5,1} 
(under nominal conditions). In order to apply the formu- 



lation of the filter in Sec. Ill to this example, we assume 
weighting matrices L = 5, H = 3. We apply the max- 
plus deterministic filter to this example system using a 
filter horizon of 5 time steps. Assuming the case of a 
systemic failure of all but one of the sensors - i.e. the 
set of C j becomes {0.01,0.1,0.2,0.01,1}, we obtain 
the results indicated in Fig. [T] for the case of no errors 
in the initial state assumption and Fig. [2] for the case of 
an initial state error (in the estimate). 

Thus it is seen that the estimation mechanism solves 
the sensor selection problem in a robust manner. The 
algorithm uses the sensor which provides measurement 
signals which are used to minimize the cost function 
(which thereby yields a state estimate). It can be seen 
that the approach recovers the state despite noisy mea- 
surements and sensor failures. A salient feature of the 
behavior of the estimation approach is that in both 
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Fig. 1: The case of zero initial estimation error 
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(b) Sensor selected 

Fig. 2: The case of an initial estimation error 



scenarios studied in the example (zero/non-zero initial 
estimation error), when the noise in the functioning 
sensor increases rapidly over a short time duration, 
the algorithm temporarily switches to a different sensor 
which yields a lower cost function. Once the functioning 
sensor yields a better performance, it is once again used 
for estimation as expected. 

V. Conclusions and Future Directions 

In this article we introduced the application of the 
deterministic filtering concept to sensor selection for 
robust state estimation. The resulting optimal control 



problem yielded a HJB equation which was solved via 
the max-plus approach which utilizes the linearity of 
the dynamic programming propagator in order to obtain 
an efficient method to solve the HJB equation at each 
time step. The technique yielded promising results when 
applied to the case of drastic sensor failures for a linear 
system. 

Some fruitful directions along which further research 
into this topic may be directed are: the extension of 
these methods to solve the sensor scheduling problem 
for nonlinear systems, to understand the behavior of the 
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approach during short duration failures in sensors, the 
case of unmodeled system dynamics, incorporation of 
prior belief regarding sensor failure rates into the algo- 
rithm, the fusion of information from multiple sensors 
when not all of them satisfy the observability condition, 
the case where switching costs are imposed on changes 
in the sensor vector etc. One other facet of this approach 
to sensor scheduling is that it benefits from a non- 
zero input signal u(-). Thus it is useful to understand 
the requirement for persist excitation like conditions on 
the input to the system, in order to ensure robust state 
estimation. 
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